chore: Add integration tests for Android #1146
Conversation
tustanivsky
force-pushed
the
chore/android-tests
branch
from
3ba125b to
d62218d
Compare
tustanivsky
force-pushed
the
chore/android-tests
branch
from
a3d3766 to
35862ce
Compare
tustanivsky
force-pushed
the
chore/android-tests
branch
4 times, most recently
from
41221d6 to
3bc3f33
Compare
tustanivsky
force-pushed
the
chore/android-tests
branch
from
963fa85 to
3b14ec3
Compare
Fix artifact download Shell Fix Fix Fix Fix Set config Fix syntax Sauce skip run Update appium Test test test apk name Test Test Test Test Test Test Test Test Test Test Test tets Test Test Poll session status Test Test Test 2 runs Test Test Try fix redirect Test Test fix id Test app status poll Test Clean up
tustanivsky
force-pushed
the
chore/android-tests
branch
from
d8ab932 to
c9079bc
Compare
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
limbonaut
left a comment
limbonaut
left a comment
There was a problem hiding this comment.
I can't say I understand the nuances behind Sauce Labs / Appium APIs. The test script looks fine. It's a shame that crash testing doesn't work. I left a few comments that may be helpful.
| # RUN 1: Crash test - creates minidump | ||
| # ========================================== | ||
| # The crash is captured but NOT uploaded yet (Android behavior). | ||
| # TODO: Re-enable once Android SDK tag persistence is fixed (`test.crash_id` tag set before crash is not synced to the captured crash on Android) |
There was a problem hiding this comment.
Do you know if there is an issue to track?
There was a problem hiding this comment.
| # RUN 2: Message test - uploads crash from Run 1 + captures message | ||
| # ========================================== | ||
| # Currently we need to run again so that Sentry sends the crash event captured during the previous app session. | ||
| # TODO: use -SkipReinstall to preserve the crash state. |
There was a problem hiding this comment.
In Godot tests, I've added a "crash-send" command, which basically launches, initializes Sentry, waits like 10 iterations, and then quits. Maybe we could run a similar test action here after each run for relevant platforms like Android and Cocoa. This would help catch crash reports for each run, even if an App crashes on exit, for example.
| Write-Debug "App state: $appState (elapsed: $([int]((Get-Date) - $startTime).TotalSeconds)s)" | ||
|
|
||
| # State 1 = not running, 0 = not installed | ||
| if ($appState -eq 1 -or $appState -eq 0) { |
There was a problem hiding this comment.
Remark: This is not much to work with. I was hoping Sauce Labs would give a bit more information, like if the application actually crashed or exited cleanly.
| # RUN 1: Crash test - creates minidump | ||
| # ========================================== | ||
| # The crash is captured but NOT uploaded yet (Android behavior). | ||
| # TODO: Re-enable once Android SDK tag persistence is fixed (`test.crash_id` tag set before crash is not synced to the captured crash on Android) |
There was a problem hiding this comment.
Do you think we can grab crash ID from sentry-native output in app-runner?
There was a problem hiding this comment.
I’m not sure the app-runner is the right place for this. Parsing the app’s output seems test-specific and each gaming SDK that will be using app-runner may have its custom logic for test checks.
This PR adds Android integration tests for the Sentry Unreal SDK using PowerShell's Pester framework. The tests validate end-to-end functionality by running the
SentryPlaygroundsample application on mobile devices and verifying that events are properly captured and sent to Sentry.It includes the necessary CI workflows and test scripts for both local (adb) and cloud (SauceLabs) execution.
The motivation for using SauceLabs Real Device Cloud to run tests in CI pipeline is the current limitation of Android emulators which do not support the rendering backends required by Unreal applications.
Depends on:
Changes
Integration.Tests.Android.ps1for running tests locally viaadbor on SauceLabs devices.ci.ymlto include the new Android test workflow.integration-test-android.ymlto run Android integration tests using SauceLabs in CI.README.mdwith comprehensive instructions for running Android tests locally and via SauceLabs, including required environment variables and prerequisites.SAUCE_USERNAMEandSAUCE_ACCESS_KEYsecrets to the repo.Related to:
Closes #964
#skip-changelog